UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory

摘要

In this work we introduce a convolutional neural network (CNN) that jointlyhandles low-, mid-, and high-level vision tasks in a unified architecture thatis trained end-to-end. Such a universal network can act like a `swiss knife'for vision tasks; we call this architecture an UberNet to indicate itsoverarching nature. We address two main technical challenges that emerge when broadening up therange of tasks handled by a single CNN: (i) training a deep architecture whilerelying on diverse training sets and (ii) training many (potentially unlimited)tasks with a limited memory budget. Properly addressing these two problemsallows us to train accurate predictors for a host of tasks, withoutcompromising accuracy. Through these advances we train in an end-to-end manner a CNN thatsimultaneously addresses (a) boundary detection (b) normal estimation (c)saliency estimation (d) semantic segmentation (e) human part segmentation (f)semantic boundary detection, (g) region proposal generation and objectdetection. We obtain competitive performance while jointly addressing all ofthese tasks in 0.7 seconds per frame on a single GPU. A demonstration of thissystem can be found at http://cvn.ecp.fr/ubernet/.